chore: playwright by kyle-ssg · Pull Request #6562 · Flagsmith/flagsmith

kyle-ssg · 2026-01-20T17:27:09Z

I have read the Contributing Guide.
I have added information to docs/ if required so people know about the feature.
I have filled in the "Changes" section below.
I have filled in the "How did you test this code" section below.

Changes

Playwright Test Improvements

Faster Tests

Inputs are really fast compared to TestCafe
Tests are passing in CI around 2m30s vs ~4m

Better Local Development

Tests are re-runable via ui (you'll have to run npm run test:teardown before clicking play again)
Tests come with videos and clickable traces (these get uploaded from GitHub on failures too)
Tests can be ran very quickly and can be repeated (E2E_REPEAT=x) and exit on first failed test (E2E_RETRIES=0)

Added Claude Commands

/e2e - Run all tests (OSS + Enterprise)
/e2e-oss - Run OSS tests, auto-fix failures, re-run until passing
/e2e-ee - Run Enterprise tests, auto-fix failures, re-run until passing
/e2e-create - Create a new test following existing patterns

Note: You can have the tests repeat to guarantee no flakiness

/e2e 5 → Runs all tests with E2E_REPEAT=5 (runs once, then 5 more times if passing)
/e2e-oss 5 → Same for OSS tests only
/e2e-ee 5 → Same for enterprise tests only

HTML Reports

Interactive dashboard with search/filter
Test timeline and duration metrics
Detailed error messages with stack traces

e.g. this is a downloaded report, we can access reports of failures as well as videos / interactive traces.

CI/CD Features

GitHub Integration

Sticky PR comments with failure summaries
Direct links to ZIP artifacts containing:
HTML report
Videos of failures
Interactive traces
Screenshots
Automatic artifact cleanup

How did you test this code?

No code changes, all e2e refactoring
Run local image and keep track of any failures

…ponentise-feature-filters

…eature-state-view

Co-authored-by: Zaimwa9 <wadii.zaim@flagsmith.com>

…rs' into chore/componentise-feature-filters

# Conflicts: # frontend/web/components/pages/UserPage.tsx

frontend/e2e/slack-e2e-reporter.ts

frontend/e2e/tests/project-test.pw.ts

frontend/e2e/slack-e2e-reporter.ts

.github/workflows/.reusable-docker-e2e-tests.yml

The CI workflow sets E2E_RETRIES=2 but the Makefile only forwarded E2E_CONCURRENCY to the container, so retries defaulted to 1. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

frontend/e2e/slack-e2e-reporter.ts

frontend/Makefile

talissoncosta · 2026-02-27T16:49:11Z

Great work on the Playwright migration @kyle-ssg ! The DX improvements (trace viewer, DOM snapshots, interactive UI mode) are huge.

I tested extensively using both /e2e and make test. Here are my findings:

Test Results

Using make test with --grep @oss:

10 passed ✅

Using make test with --grep @enterprise:

4 passed, 2 failed ❌ (change-request-test, roles-test)

Using /e2e (all tests):

14 passed, 2 failed ❌ (same tests)

Observations

1. Enterprise tests fail against OSS API

Two tests consistently fail locally:

change-request-test → POST /create-change-request/ returns 405
roles-test → POST /roles/ returns 405

Root cause: docker-compose-e2e-tests.yml uses target: oss-api, but these tests require enterprise-only endpoints.

2. Concurrency mismatch

.claude/commands/e2e.md uses E2E_CONCURRENCY=20 → causes browser crashes locally
Makefile uses E2E_CONCURRENCY=3 → stable

3. waitForTimeout() anti-patterns

Found instances in e2e-helpers.playwright.ts, though e2e.md states "NEVER use page.waitForTimeout()".

CI handles OSS/Enterprise correctly

Looking at platform-pull-request.yml:

Job	API Image	Tests
`run-e2e-tests`	`oss-api`	`--grep @oss`
`run-e2e-tests-private-cloud`	`private-cloud-unified`	`--grep "@oss\|@enterprise"`

Can't build private-cloud locally

Tried changing to target: private-cloud-unified but build fails - requires github_private_cloud_token for private repos.

Questions

Is --grep @oss the intended way to run tests locally?
Should docs/commands default to OSS-only tests for local development?
Should we align concurrency settings? (Makefile uses 3, /e2e command uses 20)

None of these are blockers. The PR is really good and ready to merge! I just wanted to raise these observations for awareness and to clarify the expected local development workflow.

Zaimwa9

🚀 🚀

…ywright # Conflicts: # frontend/e2e/tests/project-test.pw.ts

frontend/e2e/slack-e2e-reporter.ts

# Conflicts: # frontend/e2e/tests/flag-tests.ts # frontend/package-lock.json

frontend/e2e/slack-e2e-reporter.ts

frontend/e2e/tests/project-permission-test.pw.ts

frontend/e2e/slack-e2e-reporter.ts

frontend/e2e/tests/project-permission-test.pw.ts

frontend/e2e/run-with-retry.ts

frontend/e2e/slack-e2e-reporter.ts

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

frontend/e2e/slack-e2e-reporter.ts

frontend/e2e/run-with-retry.ts

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

frontend/e2e/slack-e2e-reporter.ts

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

frontend/e2e/slack-e2e-reporter.ts

kyle-ssg and others added 30 commits September 30, 2025 18:42

fix identifier

162fceb

Fix identityName param

e52f9f9

Fix edit permission check

0ba5c15

Componentise feature filters

80811d5

Fix segment override logic

a5dc772

Fix identity override logic

4d340e3

Fix identity override logic

3ee508e

Merge branch 'chore/componentise-feature-override-row' into chore/com…

7971ac7

…ponentise-feature-filters

QA

657fd59

Merge branch 'chore/componentise-feature-override-row' into chore/com…

ce67965

…ponentise-feature-filters

Make filtering consistent across user and features page

12eee3a

simplify filters

f061a48

Merge branch 'main' into chore/componentise-feature-override-row

2939e21

Merge branch 'chore/componentise-feature-override-row' into chore/com…

e441e03

…ponentise-feature-filters

re-add loading logic

10719cb

Simplify loading logic

b790d58

Don't assume code references exist

17ea4ba

Don't assume code references exist

772e147

fix text wrapping

f3ba094

Fix tag filter in url

7b18f28

Merge branch 'chore/componentise-feature-filters' into feat/segment-f…

b4724ca

…eature-state-view

Update frontend/web/components/feature-page/FeatureFilters.tsx

eb0be4b

Co-authored-by: Zaimwa9 <wadii.zaim@flagsmith.com>

Remove releasePipelines

6174bd5

Merge remote-tracking branch 'origin/chore/componentise-feature-filte…

9b562f6

…rs' into chore/componentise-feature-filters

remove sortToHeader

dbb085a

Merge branch 'main' into chore/componentise-feature-override-row

f7e3cff

# Conflicts: # frontend/web/components/pages/UserPage.tsx

Claude context

ba3b42e

Claude context

9746d13

Claude context

5955f25

Add api-sync / optmimise contexts

efedbdc

kyle-ssg added 2 commits February 25, 2026 10:31

Fix webpack config

2116b61

Fix sentry dependency issue

f738f40

cursor bot reviewed Feb 25, 2026

View reviewed changes

frontend/e2e/slack-e2e-reporter.ts Show resolved Hide resolved

frontend/e2e/tests/project-test.pw.ts Show resolved Hide resolved

Remove duplicate waits

4a8b838

cursor bot reviewed Feb 25, 2026

View reviewed changes

frontend/e2e/slack-e2e-reporter.ts Show resolved Hide resolved

.github/workflows/.reusable-docker-e2e-tests.yml Show resolved Hide resolved

kyle-ssg and others added 2 commits February 25, 2026 11:02

Pass E2E_RETRIES from CI into Docker container

3847ebe

The CI workflow sets E2E_RETRIES=2 but the Makefile only forwarded E2E_CONCURRENCY to the container, so retries defaulted to 1. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

chore: add feature tests (#6652)

95a4716

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

cursor bot reviewed Feb 25, 2026

View reviewed changes

frontend/e2e/slack-e2e-reporter.ts Show resolved Hide resolved

frontend/Makefile Outdated Show resolved Hide resolved

talissoncosta previously approved these changes Feb 27, 2026

View reviewed changes

Zaimwa9 previously approved these changes Mar 2, 2026

View reviewed changes

kyle-ssg added 2 commits March 3, 2026 22:50

Fix a couple edge cases

056b1ae

Merge remote-tracking branch 'origin/chore/playwright' into chore/pla…

6326b8f

…ywright # Conflicts: # frontend/e2e/tests/project-test.pw.ts

cursor bot reviewed Mar 3, 2026

View reviewed changes

frontend/e2e/slack-e2e-reporter.ts Show resolved Hide resolved

kyle-ssg added 3 commits March 4, 2026 09:47

Fix edge cases

cac47c3

Merge branch 'main' into chore/playwright

1044e28

# Conflicts: # frontend/e2e/tests/flag-tests.ts # frontend/package-lock.json

Fix edge cases

556d372

cursor bot reviewed Mar 4, 2026

View reviewed changes

frontend/e2e/slack-e2e-reporter.ts Show resolved Hide resolved

frontend/e2e/tests/project-permission-test.pw.ts Outdated Show resolved Hide resolved

Improve segment change request test check

46e852a

cursor bot reviewed Mar 4, 2026

View reviewed changes

frontend/e2e/slack-e2e-reporter.ts Show resolved Hide resolved

frontend/e2e/tests/project-permission-test.pw.ts Outdated Show resolved Hide resolved

frontend/e2e/run-with-retry.ts Show resolved Hide resolved

Improve assertions

1437880

cursor bot reviewed Mar 4, 2026

View reviewed changes

frontend/e2e/slack-e2e-reporter.ts Show resolved Hide resolved

Improve assertions

277d891

cursor bot reviewed Mar 4, 2026

View reviewed changes

frontend/e2e/slack-e2e-reporter.ts Show resolved Hide resolved

frontend/e2e/run-with-retry.ts Show resolved Hide resolved

Improve assertions

b8a1a2f

cursor bot reviewed Mar 4, 2026

View reviewed changes

frontend/e2e/slack-e2e-reporter.ts Show resolved Hide resolved

kyle-ssg added 2 commits March 4, 2026 16:12

Improve assertions

cef8639

Remove realtime from flagsmith sdk in e2e mode

641e7b6

cursor bot reviewed Mar 4, 2026

View reviewed changes

frontend/e2e/slack-e2e-reporter.ts Show resolved Hide resolved

talissoncosta approved these changes Mar 4, 2026

View reviewed changes

Conversation

kyle-ssg commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Playwright Test Improvements

Faster Tests

Better Local Development

Added Claude Commands

HTML Reports

CI/CD Features

How did you test this code?

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

talissoncosta commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results

Observations

CI handles OSS/Enterprise correctly

Can't build private-cloud locally

Questions

Uh oh!

Zaimwa9 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kyle-ssg commented Jan 20, 2026 •

edited

Loading

talissoncosta commented Feb 27, 2026 •

edited

Loading